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ABSTRACT 

These proceedings report on a seminar held to 
determine the needs and wants served by the Association for 
Institutional Research (AIR) in regard to products and services 
provided by AIR to institutions and individuals. A 
vulnerability/opportunity audit was conducted to identify the 
strengths and weaknesses of the organization. Participants lifted the 
strengths of AIR in regard to support to professionals, institutional 
research (IR) as a profession, and higher education. They also 
identified the weaknesses of AIR as an institution, as well as 
social, technological, economic, and political factors contributing 
to the declining importance of AIR. Participants also analyzed the 
effects of two potentially critical events on the organization: (1) 
increasing use of the Internet as a means of networking, holding 
virtual conferences, and exchanging information; and (2) the 
development of vertical market computer software for basic IR 
functions. (MDM) 
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The Vulnerability/Opportunity Audit 



Preconference Seminar 



Institutional Vulnerability Audit 
Proceedings 



Abstract 

This is a proceedings of the 1995 AIR Preconference Seminar on the Institutional 
Vulnerability Audit. It consists of a report generated via several exercises described in the 
seminar handbook — AIR strengths, potential vulnerabilities, an analysis of the probability 
and impact of two vulnerabilities expressed in the form of event statements, the 
implications of that analysis for AIR, and recommendations we make to AIR based upon 
these implications. 

Time did not allow us to view tlie strengths and vulnerabilities of AIR for more than the 
"needs and wants served" category. To conduct a complete audit of the organization, we 
would need an additional day. In conducting the analysis of potential events, we discovered 
it meaningful to view the positive as well as negative impact of events on the organization. 
Consequently, the tide of the proceedings was changed to reflect this innovation. 
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Institutional Vulnerability/Opportunity Audit 

Every organization can be hurt by outside forces or conditions over which it has 
little or no control. Planning as practiced in many colleges and universities today 
is often deficient because it does not identify a number of potentially threatening 
forces or conditions. In the past, many of these blind spots were overlooked 
because the dominant planrung goals focused on growth. Threats were viewed 
primarily in terms of potentially damaging changes in the market Another 
planning weakness is the tendency of people to assume continuation of past trends 
without examining the likelihood of discontinuities affecting the direction of those 
trends. 

The organizational vulnerability/opportunity audit, originally developed as 
vulnerability analysis by SRI (1977), is a strategic planning tool that surfaces 
subtie or overlooked threats that may adversely or positively affect the 
organization's future. The focus is first on those threats that fall outside the 
normal sphere of organizational actions. However, often a threat can also be an 
opportunity, an opportunity that may not have surfaced without first focusing on 
threats and vulnerabilities. Hence the name, vulnerability/opportunity audit 

Vuhierability/opportunity audits are based upon five assumptions: (1) 
organizations exist because they serve some need; (2) organizations rely on 
support from the environment in which they are expected to operate; (3) organiza- 
tions are vuhierable to the changing external environment; (4) some of these 
vulnerabilities can also be opportunities, and (5) organizations are undergirded by 
the following: needs and wants served by the organization, resources and assets 
the organization relies upon, stability of costs relative to competition, the organiza- 
tion's target customer base, technologies, special abilities, corporate identity 
symbols, barriers to competition, social values important to continuance of the 
organization, sanctions, perceived integrity of the organization, and the availability 
of complementary products or services. 

The organizational vulnerability/opportunity audit reviews these supports from the 
perspective of current strengths. The supports are turned into questions: What 
needs and wants does the organization meet? What resources does it rely on? 
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What cost advantages are available? What special abilities does the organization 
have? What technologies underpin it? 

The vulnerability/opportunity audit then examines the supports' vubierability to 
removal, alteration, or substantial disruption. The key question embodied in this 
tool is: "What supportive elements, if suddenly taken away, might seriously 
impair or even destroy the organization?" 

Needs and Wants Served by AIR: Strengths 

The initial step in the process is to identify as many supports on which the 
organization depends as possible and then identify those forces, conditions, 
trends, and events that could damage these supports. A support can be tangible 
(e.g., a physical resource) or intangible (e.g., legislation, or social values that 
make a service/product desirable). 

For the seminar, we focused only on one of the categories: needs and wants 
served by the Association for Institutional Research that underpin the demand for 
AIR's products and services. 

We used the nominal group technique (NGT) to identify the needs and wants 
served by AIR. Participants were given 10 minutes to think how AIR served 
society and asked to write down their thoughts. We used a round robin approach 
where each participant in turn was asked to state how AIR served needs and wants 
in society. Only one nomination was given by each participant Each statement 
was typed on a computer and projected via a LCD panel to a screen so that all 
could see the nominations. The next person was asked to submit their "best" 
candidate; during this time the only person talking was the person nominating a 
statement; all others were requested to think about the statement to see if it 
stimulated an idea that they had not had before. 

This process went on until there were no more nominations. We then went back 
over teach nomination to clarify, discuss, edit, remove redundancies. The value of 
this process is first to have participants think before talking, and to get the 
thinking of all people in the groups. 
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The needs and wants that AIR serves as identified by the group were as follows: 
For IR Professionals 

1. Provides opportunities for global networking with fellow professionals and 
with leaders in intersecting fields 

2 . Provides a forum for testing IR member's ideas 

3 . Provides professional development opportunities for members 

4. Provides encouragement and opportunity for AIR members to test ideas on 
informed and sympathetic audience 

5. Promotes professional satisfaction 

6. Contributes to wellness (e.g., families accompany members at meetings, 
provides opportunity for social interaction, rest, relaxation, break in routine, 
interacting with similarly isolated individuals) 

7 . Initiates new members into the profession 

8 . Offers professional validation for its members 

9. Promotes awareness of career opportunities in IR 

For IR as a profession 

1 . Contributes to developing appropriate IR theory, methodologies, standards, 
procedures, and practices 

2 . Validates IR as a profession 

3 . Recognizes and promotes important work in the field 

4. Invigorates the profession 

5. Identifies critical resources (e.g., people, studies, etc.) 
For higher education 

1 . Provides information, analysis, and interpretation to external decision-makers 
(e.g., governmental leaders, accrediting agencies, stakeholders, opinion 
leaders), thereby improving the quality of their policies/decisions relating to 
higher education 

2 . Lobbies for the benefit of the membership and for higher education 
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3. Provides a professional counterpoint for policy declarations of political 
agencies 

4. Coordinates efforts to address common threats 

5 . Promotes efficient and effective delivery of education 

6. Catalyst for causing shifts in higher education paradigms by communicating 
new models of thought and patterns of behavior 

7. Activities (e.g., publications, listserv, electronic AIR, conferences, seminars) 
inform educational leaders about the educational environment in which they 
exist and may exist in the future 

8. Promotes cooperation/competition with such organizations as CAUSE, 
EDUCOM, and SCUP 

9. Provides a channel for external organizations (e.g., NCES) to communicate 
matters affecting the profession or higher education 

10. Provides network for development of studies and interinstitutional 
collaboration 

1 1 . Provides for peer interinstitutional collaboration and benchmarking 



Needs and Wants Served by AIR: Vulnerabilities 

The next exercise was to identify the vulnerabilities that AIR may have in serving 
the needs and wants of IR members, the profession, higher education, and the 
larger society. Relevant vulnerability questions asked were: 

a. Could a need or want be served some other way by some other organization? 

b. Could the need for a product or service disappear? If so, under what circum- 

stances? 

We again used the nominal group technique to identify the following 
vulnerabilities that AIR may have in this area. In order to prepare these 
vulnerabilities for further analysis, we edited the vulnerability statements into 
event statements and categorized them into vulnerabilities that would affect AIR 
directly as an organization and potential events in the social, technological, 
economic, and political macroenvironment that would have an effect on AIR if 
they should occur. These event statements are as follows: 
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AIR as an organization 

1. Other national organizations (e.g., SCUP, CAUSE, EDUCOM, AACRAO, 
AAHE, and other organizations at One DuPont Circle) subsume functions of 
AIR 

2 . Regional or state IR organizations subsume functions of AIR 

3 . Other regional or state organizations subsume functions of AIR 

4. AIR survey reveals that IR functions diffused throughout 50% of all colleges 
and universities 

5. 30% membership decline in AIR 

6. Graduate programs in higher education train 80% of new IR professionals 

7. 50% of AIR Forum evaluation indicates dissatisfaction with quality of 
presentations 

8 . 50% of IR offices required to do policy analysis 

9 . AIR leadership group becomes ossified in its thinking 

Social 

1 . 25% of high school graduates with SATs over 1200 enters labor market via 
corporate training 

2 . 50% of FIE based on Internet, distance education sources 

3 . Development of international education standards 

Technological 

1. 80% of IR professionals use the Internet as their primary means of 
networking, virtual conferences, information exchange, etc. 

2 . Artificial intelligence replaces basic IR function 

3 . Vertical market software for basic IR functions developed 

4. Software program developed that combines current data requirements by 
SPRE, IPEDS, Student Right to Know, NCES, Title Four Recertification 
Data, and accreditation coorduiated by Federal government 

Economic 

1 . 80% of IR offices have travel budgets eliminated for national conferences 
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2 . Federal funding for education reduced by 20% 

3. 60% of states reduce funding for education by 20% or more 

4. Major recession 

5 . 70% of IR functions outsourced 

6 . 65% of colleges and universities take a 5% or more b'ldget cut 

7 . US and Japan have trade war 

8 . World Trade Organization stops US and Japan trade war 
Political 



1 . "Contract with America" is operationalize 

2 . Recission bill enacted 

3 . Federal government eliminates IPEDS 

4 . US Department of Education eliminated 

5 . SPRE is eliminated 

5 . Federal financial aid replaced by voucher system 

7. 50% of public institutions derive 50% of income from non-government 
sources 

8. SPRE, IPEDS, Student Right to Know, NCES, Title Four Recertification 
Data, and accreditation coordinated by Federal government 

9. Mexican revolution 

10. Anglophon Canadian providences join USA 



Probability/Impact Analysis of Two Events on AIR 

The third exercise was to prioritize the vulnerability event statements. The five 
most potentially critical events should they occur were: 

1 . 80% of IR offices have travel budgets eliminated for national conferences 

2. Vertical market software for basic IR functions developed (i.e., software 
designed foi specific function or application in higher education) 

3. Other national organizations (e.g., SCUP, CAUSE, EDUCOM, AACRAO, 
AAHE, and other organizations at One DuPont Circle) subsume functions of 
AIR 

4. 30% membership decline in AIR 
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5. SPRE, IPEDS, Student Right to Know, NCES, Title Four Recertification 
Data, and accreditation coordinated by Federal government 

At this point we divided into two groups, each group charged with conducting a 
probability impact analysis of a critical vulnerability event. We chose the top two 
events and conducted a Delphi. Each member of the group independently 
estimated the probability that the event would occur within five years and the 
degree of negative and the degree of positive impact that the event would have on 
AIR. The group then discussed the reasoning behind the disparity in estimates. A 
recorder made notes as to the factors affecting probability of occurrence, degree of 
positive impact and degree of negative impact Group discussion then focused on 
the implications of negative and positive impact, concluding with a series of 
recommendations for the AIR Board of Directors to consider when planning for 
the future. The analyses of the top two events follow. 

Event: 80% of IR professionals use the Internet as their primary means 
of networking, virtual conferences, information exchange, etc. 

Median probability event will occur within five years: 60% 

Forces affecting probability: 

1 . Availability of networking at individual institutions increasing 

2 . Internet training increasingly available 

3 . Increasing willingness of IR professionals to use Internet 

4. Motivation and reinforcers to use Internet come from 

a. state of the art 

b. speed of information 
c user friendly interface 

Degree of negative impact: very light 

Implications of negative knpact 

1 . Forum attendance may decrease 

2 . Interest in reading paper versions of manuscripts may decrease 

3 . Ownership of ideas becomes more vulnerable 
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4. Fragmentation of ideas over Internet (as opposed to complete speeches at the 
Forum) 

5 . People respond immediately; less thoughtful exchanges 
Degree of positive impact: very high 

Implications of positive impact: 

1 . Timely sharing of information 

2 . Increased productivity 

3 . Increased possibilities for institutional collaboration 

4 . Opportunity for AIR to sponsor Internet training 

5 . Pre-forum information/papers can be posted electronically; saves paper, and 
changes Forum sessions from passive, listening to presentations, to active 
discussions led by presenter. 

6. Increase Forum attendance (People who interact electronically will seek 
opportunities for face-to-face meetings) 

What should AIR do? 

1 . Maximize use of Internet in conducing AIR business, initiating professional 
development opportunities, and fulfilling AIR mission. 

2. Create a WEB site; put AIR publications, including Forum papers, on Web 
site 

3. Explore how other professional associations are meeting the 
challenge/opportunities of the Internet 

4. Survey membership for their Internet capabilities and needs in the areas of 
technology, institutional and personal 

5 . Monitor policies affecting Internet access and use 



Event: Vertical market software for basic IR functions developed (i.e., 
software designed for a specific function or application in higher 
education) 

Median probability event will occur within 5 years: 30 
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Forces affecting probability 

1 . Current changes in information technology (e.g., more powerful software and 
hardware engines) increase probability 

2. The more complex it is (i.e., the greater the scope of functions incorporated), 
the li^ss the probability 

3. The diversity of types and sizes of colleges and universities increases the 
scope and decreases the probability 

4. Possibility that the Federal government will require a standard methodology of 
reporting required data increases the probability 

6 . The greater the cost of product, the less the probability of broad adaptation 

7. If decision-makers see a substantial increase of functionality over tools 
currently in use, the likelihood that the software will be developed is increased 

Degree of negative impact: Slightly above light impact 

Implications of negative impact 

1 . Perception that with this supertool, colleges will need fewer IR personnel 

2. Potential conflict with other professional groups within institutions over 
control and implementation of the software (e.g., ?) 

Degree of positive impact: Light to moderate 

Implications of positive impact 

1 . Need to train IR professionals how to use the new software 

2. Need to coordinate use and output between institutions and the agencies to 
whom they report 

3. New opportunities for doing higher level work (can do truly comparable 
surveys presently limited by the lack of common methodologies) 

4. New opportunities for cooperation with other institutions to conduct 
benchmarking 

6. IR Offices can standardize reporting to stakeholders and the media by 
reporting criteria and methodology of report (thereby enabling, for example, 
IR offices across the country to provide comparable data to organizations such 
as US News and World Report). 

What should AIR do? 
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1 . Facilitate, promote, encourage more sophisticated software for IR reporting 

2. Test and validate such software (i.e., AIR should influence the practicality, 
accuracy, validity and appropriateness), and assure that the software should 
not be driven by political forces. 

3 . Provide training and the use of the software 

4 . Develop information-brokering as a spin-off profession within IR 

5 . Identify new IR opportunities for this new tool. 

6. AIR should hold 

a. a contest for developing the software 

b . facilitate collaboration of IR professionals to develop the software 



Discussion 



We were able to cover only one of the underpinning categories, needs and wants 
served, and were also limited to an analysis of only two events that surfaced in 
this category. Even so, we were able to experience how to conduct the audit and 
to derive several recommendations that may be useful to the AIR board when 
planning for the future. 

The orggnizational vuberability/opportunity audit is not without limitations. For 
example, the process does not address those problems due to the internal 
environment or those problems due to competitor's actions. Too, team members 
may not have sufficient knowledge or vision to evaluate threats. 

But there are a number of benefits to applying an organizational 
vulnerability/opportunity audit First, the process increases the expertise and 
judgment of organizational leaders by making them aware of conflicts in planning 
assumptions-both implicit and explicit This awareness, in turn, helps 
organizational leaders anticipate change and surface emerging issues. Moreover, 
by identifying overiooked threats (environmental, not those posed by 
competitors), leaders have time to monitor threatening situations, to review 
options, and to prepare contingency plans, thereby avoiding unpleasant surprises. 

The auditing process itself focuses attention on factors and issues most relevant to 
the institution. Some organizations try to monitor everything in the environment. 
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which wastes both time and resources. Other organizations monitor some 
environmental areas, not necessarily the right ones, wasting money and exposing 
the organization to unnecessary risks. The organizational vulnerability/opportunity 
audit exercise focuses on the most important areas, the ones that may cause the 
biggest problems for the institution. 
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